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Abstract 

Despite the growing popularity of robotics competitions such as FLRST LEGO League, robotics 
activities are typically not found in regular K—12 classrooms. We speculate that, among other 
reasons, limited adoption is due to the lack of empirical evidence demonstrating the effect of 
robotics activities on curricular goals. This paper presents a mixed methods study exploring 
the impact of a summer robotics camp on middle school students’ physics content knowledge 
and scientific inquiry skills. It was found that the camp enhanced students’ physics content 
knowledge but failed to improve their skills in conducting scientific inquiry. Qualitative data 
provided an explanation of the findings. (Keywords: educational robotics, scientific inquiry, 
problem-based learning, constructionism.) 

INTRODUCTION 

Recent years have witnessed an increasing interest in the educational use of 
robotics. International competitions such as RoboCup (The RoboCup Federa- 
tion, 2006), FIRST LEGO League (2006), and RoboFesta (2006) have at- 
tracted children and young adults around the world to participate in various 
challenges. Many universities and schools have been offering robotics summer 
camps and enrichment programs to K—12 students. The commercial market of 
educational robotics is also growing. Recent research by Japan Robotics Asso- 
ciation, United Nations Economic Commission, and the International Federa- 
tion of Robotics (cited in Kara, 2005) indicates that the market growth for per- 
sonal robots, including those used for entertainment and educational purposes, 
has been tremendous. This trend may continue in the next several decades. 

Despite their growing popularity, robotics activities are usually not found 
in regular K—12 classrooms. One of the reasons might be that there is limited 
empirical evidence to prove the impact of robotics activities on curricular goals. 
Most of the literature on robotics use in education is anecdotal and descriptive 
in nature (e.g.. Ford, Dack, & Prejean, 2006; Lau, Tan, Erwin, & Petrovic, 
1999). A few exploratory studies have revealed positive perceptions of the 
educational value of robotics activities among teachers and students (Petre & 
Price, 2004; Robinson, 2005). This is encouraging, but measurable evidence 
is needed to convince educators of the positive impact of robotics activities on 
curricular goals. 
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Educational theorists such as Papert (1993) believe that robotics activities 
have tremendous potential to improve classroom teaching. Educators have 
started to generate ideas and develop activities to incorporate robotics into the 
teaching of diverse subjects, including math, science, and engineering (Bratzel, 
2005; Wang, 2004). However, without research evidence to support their direct 
impact on students’ academic performance, robotics activities may continue to 
be kept out of regular classrooms. As an effort to fill the gap in the literature, 
this article reports on a study that examines the effect of a two-week robotics 
summer camp on middle school students’ physics content knowledge and scien- 
tific inquiry skills. Findings from this study not only offer support for the edu- 
cational use of robotics to achieve curricular goals, but also provide guidance on 
how to design and implement educational robotics programs. 

REVIEW OF THE LITERATURE 

In this section, we first present constructionism as the theoretical base for 
designing robotics activities for educational purposes. We then review the lit- 
erature on science teaching and inquiry to support the use of robotics activities 
in teaching science content knowledge and scientific inquiry skills. Finally, we 
present existing studies on educational robotics to identify the gap in the 
literature. 

Constructionism 

Constructionism is a learning and instructional theory rooted in the philo- 
sophical tradition of constructivism, which emphasizes the active role of the 
learner in collaboratively constructing knowledge in a rich context (Duffy & 
Cunningham, 1996). Papert (1991) clarifies the relationship between construc- 
tionism and constructivism: 

Constructionism... shares constructivism’s connotation of learning as 
“building knowledge structures” irrespective of the circumstances of the 
learning. It then adds the idea that this happens especially felicitously 
in a context where the learner is consciously engaged in constructing 
a public entity, whether it’s a sand castle on the beach or a theory of 
the universe, (p. 2) 

Constructionism shares similarity with other constructivist theories such as 
situated cognition (Brown, Collins, & Duguid, 1989) and cognitive flexibility 
theory (Spiro, Coulson, Feltovich, & Anderson, 1988). However, construc- 
tionism is unique in that it emphasizes “learning by making” as the key aspect 
of the learning activity. The artifacts to be created are “objects to think with” 
(Papert, 1993, p. 11). They provide tangible means for learners to evaluate and 
modify their understanding of science concepts while collaboratively design- 
ing, building, evaluating, and modifying artifacts meaningful to them and the 
community (Puntambekar & Kolodner, 2005). Learning occurs when learners 
are engaged in the process of creating artifacts and communicating about their 
design. 
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Constructionism has informed a series of studies in which technology is used 
to create a context that enables “learning by making” and “learning by design.” 
Primary research projects in this area include those that use Logo programming 
in teaching mathematics (Papert, 1993), involve children in designing com- 
puter games (Kafai, 1996), engage members of a virtual community to create 
objects and space (Bruckman & Resnick, 1996), and facilitate learning with 
programmable bricks (Sargent, Resnick, Martin, & Silverman, 1996). Research 
on programmable bricks has led to the commercial product of LEGO Mind- 
storms robots (LEGO Group, 2006). 

Science Teaching and Inquiry 

Scientific inquiry is considered the centerpiece of science teaching. This belief 
is reflected in the National Science Education Standards (NSES), which posi- 
tion scientific inquiry as a core standard and stipulate that appropriate subject 
matter knowledge be incorporated in inquiry-based teaching (National Re- 
search Council, 1996). NSES defines scientific inquiry as 

. . . the diverse ways in which scientists study the natural world and 
propose explanations based on the evidence derived from their work. 
Inquiry also refers to the activities of students in which they develop 
knowledge and understanding of scientific ideas, as well as an under- 
standing of how scientists study the natural world. (National Research 
Council, 1996, p. 23) 

Inquiry skills include the abilities to conduct and understand scientific inqui- 
ry, including “asking questions, planning and conducting investigations, using 
appropriate tools and techniques to gather data, thinking critically and logi- 
cally about relationships between evidence and explanations, constructing and 
analyzing alternative explanations, and communicating scientific arguments” 
(National Research Council, 1996, p. 105). 

Robotics activities are ideal for teaching scientific inquiry skills. In inquiry- 
based learning, students need a rich context to investigate questions and de- 
velop scientific argumentation skills (Baumgartner & Reiser, 1998; Kolodner et 
al., 2003). This context is usually not available in the traditional classroom. Ro- 
botics activities may be a promising alternative. They may provide a rich con- 
text needed for students to identify and investigate problems, generate hypoth- 
eses, gather and analyze data, and to determine findings and interpret results. In 
addition, developing robots may provide an opportunity for learners to acquire 
science content knowledge since mathematics and physics are the foundation 
for engineering and programming, which are required in robotics development. 

Research on Robotics in K-12 Education 

Limited research has been conducted as to the impact of educational robotics 
activities on K-12 students’ learning. Much of the literature on educational ro- 
botics focuses on describing the activities in robotics educational programs with 
some discussion of their effectiveness based on the anecdotal evidence (e.g.. 
Ford, Dack, & Prejean, 2006; Lau, Tan, Erwin, & Petrovic, 1999). Only a few 
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studies have employed qualitative or quantitative methods to explore the impact 
of robotics activities (Barker & Ansorge, 2007; Nourbakhsh et al., 2005; Petre 
& Price, 2004; Robinson, 2005; Sklar, Johnson, & Lund, 2000). 

Both anecdotal and research literature indicates that robotics activities have 
educational values, but most of the research depends on observation and inter- 
view data without any quantitative measures of the direct impact of the activi- 
ties on students’ learning. For example, Robinson (2005) interviewed three 
science teachers who used Robolab, a Mindstorms robotics program, to teach 
8* grade physics. The teachers perceived that robotics activities may promote 
inquiry, increase students’ interest in physics, and provide opportunities for 
expanding English vocabulary for English as Second Language (ESL) students. 
Sklar, Johnson, and Lund (2000) interviewed teachers at RoboCup Junior 
2000, an international event, about their perceptions of the educational value 
of the challenges. Most of them believed in the effectiveness of RoboCup in en- 
hancing academic skills and personal development. 

Other researchers approached their studies a little differently. Instead of in- 
terviewing teachers, they focused on the learners’ perceptions. Petre and Price 
(2004) conducted interviews and observations of children attending RoboCup 
Junior (ages 6 to 18) and RoboFesta (ages 12 to 14) competitions. Children 
reported that they learned programming, problem solving, teamwork, hard- 
ware, and electronics while building and programming robots. Nourbakhsh and 
colleagues (2005) gathered comprehensive self-report data from high school 
students in a full-time, seven-week, eight-hour-per-day course on robotics. The 
primary data source was the weekly survey that students filled out anonymously. 
The initial survey that was administered on the first day of the class gathered 
data on students’ technological background, their expectations for learning 
from the class, and their plans for college and beyond. In the following weeks, 
students filled out a written survey every Monday to rate the performance of 
their team, describe the effectiveness of specific course activities, and discuss 
the problems they encountered and the discoveries they made. In the final sur- 
vey that was completed during the last week of the class, students reflected on 
what they have learned. Analysis of the surveys indicated that students found 
the class interesting and the activities effective. At the beginning of the class, 
most of them expected to learn about mechanics and programming related to 
robotics and only a small percentage of participants expected to achieve learning 
goals related to teamwork, problem solving, and self identification with science 
and technology (interest in technology, confidence to work with technology, 
and aspiration in pursuing education and careers in science and technology). 
Interestingly, at the end of the seven-week class, the majority of the students 
reported learning not only about mechanics and programming, but also about 
teamwork, problem solving, and self identification with science and technology. 

We found only two studies that went beyond gathering self-report and obser- 
vation data and used quantitative measures to evaluate the impact of robotics 
on student learning. One (Barker & Ansorge, 2007) aimed to determine the 
effects of an after school robotics science intervention on students’ (ages nine 
to eleven) achievement in science, engineering, and technology as measured by 
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a 24-item multiple-choice instrument. It was found that students in the robot- 
ics intervention improved significantly from the pretest to the posttest, whereas 
students in the control group did not improve significantly. In addition, the 
multiple-choice instrument was found to be valid and reliable in this study. 
Another study (Wagner, 1998) intended to identify whether the use of robotics 
in science curriculum has greater effect on children’s (ages eight to twelve) prob- 
lem solving skills and performance in science achievement tests than the use of 
battery powered (non-robotic) manipulatives or no manipulatives in the science 
curriculum. It was found that compared to the use of manipulatives, the robot- 
ics intervention did not significantly improve science achievement or general 
problem solving, but did improve programming problem solving. In addition, 
both robotics and battery-powered manipulatives were superior to the tradi- 
tional instruction in improving science achievement. These studies were encour- 
aging; they provided some empirical support for the use of robotics in science 
education. However, none of the studies explicitly measured scientific inquiry 
skills as an outcome of the intervention. This study attempts to close this gap 
through an effort to assess students’ acquisition of scientific inquiry skills and 
science content knowledge. 

RESEARCH PURPOSES AND QUESTIONS 

An educational research center in a southern research/ teaching university 
offered a robotics summer camp to teach physical science to middle school 
students. The camp ran for two and a half hours each day for a period of two 
weeks in the summer of 2006. 

The purposes of the study were to evaluate the impact of a robotics summer 
camp on students’ physics content knowledge and scientific inquiry skills and 
to explore various factors that might have contributed to the impact of the 
program. Findings from the study may not only provide empirical support for 
stakeholders to adopt educational robotics programs but also inform the de- 
sign and implementation of these programs. The following research questions 
guided the study: 

1 . Do student participants exit the summer robotics program with increased 
content knowledge? 

2. Do student participants exit the summer robotics program with better sci- 
entific inquiry skills? 

METHODS 

Mixed methods guided data collection and analysis for this study. As a com- 
plement to traditional qualitative and quantitative research, mixed methods re- 
search bridges the schism between the two. It is considered as the third research 
paradigm in educational research, with the first two being quantitative and 
qualitative methods (Johnson & Onwuegbuzie, 2004; Onwuegbuzie & Leech, 
2004). Mixed methods were appropriate for this study, which aimed not only to 
examine whether the summer camp led to increased physics content knowledge 
and better scientific inquiry skills, but also to explore various factors that might 
have contributed to the impact of the program. The dual purposes of the study 
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to measure and to understand the effect of the summer camp called for mixed 
methods design in which both quantitative and qualitative data were gathered 
to confirm, cross-validate, and corroborate the impact of robotics activities on 
students’ physics content knowledge and scientific inquiry skills. Quantitative 
data were gathered to answer quantitative research questions; qualitative data 
helped supplement and explain the quantitative findings. 

The Context and Participants 

The two-week robotics camp consisted of a mix of whole group and small 
group activities with hands-on work and discussions. Students worked in small 
groups to complete several challenges such as the Mars Rover Challenge, Tug- 
of-war Challenge, and Creature Bot Challenge. They used LEGO Mindstorms 
robotics kits and Robolab programming environment to complete the chal- 
lenges. At the end of each day, whole group debriefing provided an opportunity 
for groups to share ideas and show work in progress. To support students to 
complete the challenges, we provided tutorials on robotics programming, light 
sensors, and gears, and facilitated activities such as analyzing sample robotic 
toys and devices. 

Participants of the study included 2 1 middle-school students who enrolled in 
the robotics summer camp and their 10 facilitators. We sent an advertisement 
about the summer camp to various schools in the geographic area, and posted it 
in a local newspaper. The first 2 1 students who applied were admitted. Among 
the middle-school students, 18 were boys and three were girls. They were enter- 
ing sixth to eighth grade in the fall semester of 2006. Many of the students had 
familiarity with LEGO bricks, but only a few had experience with the LEGO 
Mindstorms robotics kit. Students were assigned to a two or three member 
team guided by a facilitator. Among the 10 facilitators, seven were females and 
three were males, including five teacher candidates, three doctoral students in a 
cognitive science program, and two teacher educators with K-12 experience. 

The Quantitative Methods 

A single group of 2 1 summer camp participants was pretested, exposed to the 
summer camp program, and posttested to answer the research questions. 

Measurement and Data Collection 

For each student participant, a factual knowledge measure and a scientific 
inquiry measure were given both before and after the robotics intervention. 

The physics content knowledge measure included twelve multiple-choice items 
developed by our team to assess students’ understanding of Newton’s Laws 
of Motion. The scientific inquiry measure included five questions, based on an 
instrument developed by researchers at Harvard Graduate School of Educa- 
tion (Dede & Ketelhut, 2003). In the scientific inquiry measure, students were 
provided with problem scenarios and then asked to respond to questions. For 
example, students were given this scenario: “After many observations, you find 
that your bicycle tires look flatter on cold winter days than they do on hot sum- 
mer days — even though you fill them with the same amount of air.” Then, stu- 
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dents answered three questions that required them to describe the problem in 
the scenario and to develop the hypothesis and the procedure that they would 
take to test the hypothesis. Another more complex scenario was also provided to 
students along with two questions that required them to generate a hypothesis 
and describe the procedure to test the hypothesis. 

Data Analysis 

In order to answer research question one, “Do student participants exit the 
summer robotics program with increased content knowledge?” a two tailed 
paired t-test was calculated to compare the pretest and posttest scores on the 
physics content knowledge measure. In order to answer research question two, 

“Do student participants exit the summer robotics program with better scientif- 
ic inquiry skills?” a two tailed paired t-test was calculated to compare the pretest 
and posttest scores on the scientific inquiry measure. 

The Qualitative Methods 

An ethnographic study was carried out to explain the quantitative data. In- 
sights gained from facilitator focus group interviews, individual facilitator inter- 
views, facilitator reflections, and researcher held notes helped further examine 
and explain the impact of the summer camp on students’ acquisition of physics 
content knowledge and scientific inquiry skills. 

Data Sources 

Facilitator focus group interviews. We conducted daily focus group interviews 
with facilitators at the end of each day. During the interviews, the group dis- 
cussed various issues that arose in the camp and the strategies they used to ad- 
dress the issues. Field notes were taken to record the key ideas that emerged in 
the focus group interviews. 

Facilitator interviews. We conducted individual interviews with four facilita- 
tors after the camp to understand their experience in the camp and their per- 
ceptions of students’ learning. We emailed a request for an interview to all seven 
participants who turned in reflections and four of them were able to participate 
in the interviews; thus they were a convenience sample. The interviews were 
recorded and transcribed. 

Facilitator reflections. Seven out of 10 of the facilitators submitted daily reflec- 
tions of their camp experience. Researchers sent out a list of question prompts 
to guide the facilitator reflections. These questions focused on the effect of the 
robotics summer camp on students’ learning and the effectiveness of the facilita- 
tion strategies. 

Researcher field notes. A researcher kept daily held notes of observations and 
reflections of the camp experience. The researcher was introduced to the facili- 
tators and students at the beginning of the summer camp. She was there every 
morning during the entire two weeks. At the beginning and end of each day, 
she helped the facilitators set up the equipment and clean up the classroom. 
During the camp activities, she had no responsibilities in the classroom; she fo- 
cused on data collection. 
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Data Analysis 

We imported interview transcripts, field notes, and facilitator reflections into 
NVivo 7, a software package that helps manage and analyze qualitative data. 
Miles and Huberman’s (1994) procedures guided data analysis. First, in the 
data reduction step, we coded the transcripts, reflections, and field notes into 
conceptual chunks and grouped them into categories. Next, in the data display 
step, we ran queries to make sense of the relationship among the categories. Fi- 
nally, we wrote up conclusions and verified them. Please note that pseudonyms 
are used to refer to the facilitators in this paper. 

FINDINGS 

Physics Content Knowledge 

Statistical analysis indicates a significant difference on the physics content 
knowledge measure from pretest to posttest t(20) = -5.275, p = .004 (M 8.40; 
M^ost= 9.75). That is, the robotics summer camp had a statistically significant 
impact on students’ gains in physics content knowledge. 

In this robotics summer camp, various activities might have provided op- 
portunities for students to experiment with physics concepts. For example, the 
downhill racer task allowed the students to determine the impact of various 
variables on the movement of vehicles. The variables included the weight, the 
angle of the ramp, diameters of the wheels, the friction between the wheel and 
the surface, and the location of the weight. In the robotic mouse activity, the 
students discussed how the flow of energy was changed with a “whisker” switch, 
a touch sensor that alters the direction of a robotic mouse. 

Short lessons, tutorials, and debriefings embedded in the problem solving 
activities might have helped students make the connection between experience 
and scientific concepts. For example, Madison, the lead facilitator, introduced 
Newton’s Three Laws of Motion to the whole class during the camp. In later 
debriefings when several students presented their design of a robot powered 
by balloons, they cited Newton’s Laws to explain the driving force behind the 
movement of the balloons. In a mini problem solving activity, students cre- 
ated paper devices to stay aloft as long as possible. At the debriefing, students 
discussed concepts such as kinetics, gravity, velocity, and weight to explain their 
design. A tutorial on gears might have also contributed to students’ learning of 
scientific concepts. In this activity, students moved through stations to explore 
various gears and read about their strengths and weaknesses. The tutorial en- 
couraged several groups to use gears in the design of their robots. 

Although short lessons and tutorials embedded in the robotics summer camp 
might have contributed to students’ acquisition of physics content knowledge, 
facilitators reported that students generally showed less interest in lessons and 
tutorials as compared to robotics building and programming tasks. During 
debriefing, several facilitators mentioned that these activities might be more 
interesting to students if they were offered in a more just-in-time manner. For 
example, the robotic mouse activity was meant to help students understand 
how a touch sensor works, but some students were not interested. The lead fa- 
cilitator, Madison, suggested that the activity would have been more meaning- 

208 Winter 2007-2008: Volume 40 Number 2 

Copyright © 2007, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int’l), iste@iste.org, www.iste.org. All rights reserved. 



ful if it were presented when the students had a need to use touch sensors on 
their robots. Similarly, although some groups added gears to their robots after 
going through the tutorial on gears, one facilitator commented that the tuto- 
rial might have been offered too late, because his team had already completed 
the design of the robot when they studied the tutorial. Several facilitators made 
similar comments on the robotic toy analysis activities. At the beginning of the 
summer camp, students moved through stations to analyze a variety of robotic 
toys. They were given a list of questions concerning the capabilities and features 
of the robots to guide their analysis. Although students were fascinated by these 
toys, they showed less interest in the analysis task. During facilitator debriefing, 
it was recommended that these toys should not only be presented at the begin- 
ning of the summer camp to stimulate interest, they should also be made avail- 
able throughout the summer camp to encourage students to analyze the toys to 
generate solutions to their design challenge. With a specific purpose in mind, 
students might find the robotics toy analysis more relevant. 

The impact of the robotics summer camp on students’ physics content knowl- 
edge might go beyond what was reflected in the content knowledge assessment. 
It is hoped that the summer camp experience will impact students’ future learn- 
ing of physics or other science subjects. A facilitator. Brook, reflected on the 
significance of this learning experience: 

Watching these students today helped me to realize that any type of 
experience is valuable and that the goal isn’t to create an immediate 
product. The students working today may not have understood or 
remembered the formulas or physics theories we discussed but they 
will create a memory of their experience. Later, when these students 
are in a physics or science class, they can match what is learned in the 
classroom to the experiences they have developed through this camp. 
(Brook, Daily Reflection, Day Eight) 

Another interesting finding related to content acquisition was that a couple 
of students who would be considered as low performing students in regular 
classrooms also improved in the physics content knowledge assessment. Al- 
though they were not as engaged in the activities as other students, they might 
have acquired some physics concepts from peer observations, group discussions, 
debriefings, lessons and tutorials, as well as the customized activities tailored to 
meet their special needs. For example, the robotics activities seemed to be too 
challenging for a child diagnosed with attention deficit hyperactive disorder 
(ADHD). His facilitator, Nicole, pulled him aside and asked him to complete a 
very structured robotic building task by following step-by-step instructions on a 
manual. Nicole reflected on the child’s progress in a positive tone: 

He still needed support to begin and occasionally in progress, but he 
was able to complete a couple of tasks on his own. I think this was 
great progress for him... Also note that he was FULLY engaged when 
watching what others were doing. He was OBSERVING. Something 
was happening. That is NOT a bad thing. (Nicole, Daily Reflection, 

Day Nine) 
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Inquiry Skills 

No statistically significant difference was found when comparing pretest 
and posttest scores from the scientific inquiry measure-. t(20) = -1.870,/» = .077 
(M 3.50; M = 4.28). This result initially appeared to conflict with the 
quafitative data, tor example, throughout facilitators’ reflective journals and 
interviews, their discussions of physics content knowledge were sporadic as 
compared to their numerous discussions on problem solving. Ironically, quanti- 
tative data demonstrated that students made significant progress in their physics 
content knowledge yet failed to demonstrate significant improvement in their 
performance on the assessment of scientific inquiry skills. 

Further analysis of the qualitative data related to problem solving and scien- 
tific inquiry helped us make sense of the finding. Even though the scientific 
inquiry process was introduced in the camp, students predominantly used the 
trial and error method to solve problems. For example. Brook described how 
her students failed to go through the scientific inquiry process. Instead, they 
worked through an intuitive procedure by trying various ideas for the alert 
system and addressing problems as they arose. Joshua stated that his students 
either skipped some of the inquiry steps, or did not have a systematic plan for 
isolating and testing variables to find the solution. 

Several factors might have contributed to students’ failure to follow the scien- 
tific inquiry process. First, the excitement and novelty of building robots might 
have rendered the design challenges less appealing. Some students had difficulty 
focusing on any specific design goals or problems. Brook was frustrated that 
her students kept ignoring the design challenges and the scientific procedure. 
Instead, they built their robots without a clear sense of what they wanted to 
achieve. Justin suggested providing an opportunity for students to explore the 
LEGO robotics kit before requiring them to work on a design challenge. They 
might be ready to solve the problem “after the novelty of new materials and a 
new project were over.” 

Second, the difficulty of some challenges might not have encouraged explicit 
application of the scientific inquiry process. During the follow-up interview, 
Joshua described how some robotics activities might not be challenging enough 
to require explicit thinking on every step of the scientific inquiry process. He 
stated. 

He almost saw exactly what it needed to be, and all the steps in between 
he just didn’t see as necessary. It is kind of hard to tell somebody to 
slow down, take it step by step, whenever he knows it is just a matter 
of intuition... (Joshua, Follow-up Interview) 

Third, the development of the scientific inquiry skills might be a long-term 
process that requires much more than a two-week summer camp. For example, 
Justin reflected that acquisition of scientific inquiry method might be a gradual 
process; it might take students years to integrate scientific inquiry into their 
problem solving process. Madison had similar thoughts. She wondered whether 
students’ inability to use the scientific process was influenced by their level of 
development. 
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Fourth, facilitators in the camp might not have had enough knowledge and 
skills related to the scientific inquiry process to facilitate its use. In teachers’ re- 
flective journals, only Madison specifically mentioned the term scientific inqui- 
ry process, and what the process entails. Does this indicate that the facilitators 
themselves might not have had a deep understanding of the scientific inquiry 
process? Their lack of knowledge and skills in the scientific inquiry process 
might have impacted their ability to encourage and model the use of the process 
in the design challenges. 

The last possible explanation of the non-significant finding was that the scien- 
tific inquiry assessment may not be robust enough to discern a difference. The 
Harvard research group who created the scientific inquiry instrument used in 
this study recently questioned its validity (Ketelhut & Dede, 2006). In a new 
study, they added another measure in which students wrote an authentic report 
in the format of a letter to describe their hypothesis, experimental design, find- 
ings and recommendations for solving a city’s health problems. Researchers 
found that students who scored low on the original survey-based measure wrote 
letters with similar quality as those written by students who scored high on the 
original measure. This finding indicates that the original measure was not sensi- 
tive enough to detect students’ learning of scientific inquiry. More authentic 
assessment of scientific inquiry skills may be needed in the future. 

How can the learning environment be structured to encourage students to 
follow the scientific inquiry process? Justin was wondering how educators could 
balance between giving students the freedom to explore but at the same time 
directing them toward the scientific process for solving problems. He wrote, 

I wonder about the issue, which seems to be coming up in most or 
all of the groups, of resistance to being methodical and to writing 
things down. I’m not sure to what extent it’s beneficial to them for 
us to try to work against this. Maybe they’re at a point where they’re 
going to learn better by being more exploratory and less structured 
— or maybe they’d do better being reined in and directed more. Of 
course there’s a balance, but I’m not sure where that balance should 
be. There’s definitely a short-term cost in working against their natural 
inclination; I’m not sure about the longer term cost/benefit. (Justin, 

Daily Reflection, Day Four) 

Quite a few facilitators described the dilemma of choosing either to let 
students follow their instinct to solve the problems or to interrupt students’ 
thought process and ask them to clarify their hypothesis and data gathering pro- 
cedure. When a child is deeply engaged in problem solving, it is difficult to stop 
them to ask for clarification. 

CONCLUSIONS 

This study found that the summer robotics camp enhanced middle-school 
students’ physics content knowledge. This finding is consistent with the litera- 
ture. In the previous studies, teachers’ perception data (Robinson, 2005; Sklar, 
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Johnson, & Lund, 2000), learners’ self-report data (Nourbakhsh et ah, 2005; 
Petre & Price, 2004), and direct measures of achievement (Barker & Ansorge, 
2007; Wagner, 1998) showed that robotics programs increased learners’ science 
content knowledge. This study provides more support for this finding. More- 
over, this study explored the various summer camp components that might have 
contributed to students’ acquisition of physics content knowledge. These com- 
ponents include activities that allowed students to experiment with various sci- 
entific concepts as well as lessons, tutorials, and debriefing sessions that helped 
students make connection between the activities and scientific concepts. 

This study also found that the summer robotics camp failed to improve stu- 
dents’ skills in conducting scientific inquiry. This finding seems to be at odds 
with results in the previous studies in which teachers and students perceived 
that robotics programs promoted inquiry and problem solving. However, we 
argue that this finding does not contradict the existing literature; rather, it adds 
to the body of knowledge related to the educational use of robotics. Although 
students and teachers in several previous studies (Nourbakhsh et ah, 2005; Petre 
& Price, 2004; Robinson, 2005; Sklar, Johnson, & Lund, 2000) believed that 
students learned problem solving from robotics programs, only one of them 
(Robinson, 2005) explicitly stated that such programs may promote scientific 
inquiry. These studies focused on a set of problem solving skills that are unre- 
lated to the scientific inquiry skills measured in our research. In the study that 
does refer to inquiry, teachers reportedly perceived that the robotics program 
may promote inquiry. This is an examination of a general belief quite different 
from this study, which measures students’ gains in scientific inquiry skills from 
one robotics summer camp. Although we agree that robotics programs may 
promote problem solving and scientific inquiry, we found that these are not 
easily achievable goals. At the end of the robotics summer camp, students’ gain 
in scientific inquiry skills was not evident in the measure that we used. It may 
require more careful design and implementation of the robotics program as well 
as better quantitative measures to document any significant gains in students’ 
scientific inquiry skills. 

IMPLICATIONS AND FUTURE RESEARCH 

This study was intended to not only provide empirical evidence for the edu- 
cational use of robotics, but also to inform the design and implementation of 
robotics programs. The findings of the study suggest the following implications 
or principles for developing and implementing robotics programs; they may 
also serve as potential topics for future research. 

Principle 1: Just-in-time resources such as lessons, tutorials, and examples 
should be embedded in robotics programs to support scientific inquiry and ac- 
quisition of content knowledge. In this study, the resources we offered enabled 
students to acquire content knowledge. They provided the concepts and prin- 
ciples that students needed to build and program the robots. However, the re- 
sources would have been more relevant and interesting if they were presented in 
a just-in-time manner. For example, the tutorial on gears might have been more 
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relevant if available earlier in the program. Robotic toy examples would have 
been more beneficial if they had been provided to serve as sample problem solu- 
tions throughout the program. The finding of the study is consistent with the 
literature. Many researchers emphasize the importance of providing resources 
such as case examples, information resources, and various scaffolding tools to 
support student-centered learning (e.g., Hannafin, Land, & Oliver, 1999; Jona- 
ssen, 1999; Kolodner et ah, 2003). Future studies are needed to develop and 
research various types of resources that are important in robotics programs. 

Principle 2: Students should be encouraged to explain their design by citing 
related scientific concepts and principles during debriefings. Researchers in the 
field of problem-based learning advocated the role of whole-class or small-group 
debriefings in providing students with an opportunity to synthesize and reflect 
on their learning (Barrows &Tamblyn, 1980; Nelson, 1999). In our study, we 
found it important for facilitators to explicitly ask students to use scientific con- 
cepts to explain the design of their robots and the strategies that worked or did 
not work for them. Explicit discussions of the scientific concepts during whole- 
group debriefing might have contributed to students’ gains in physics content 
knowledge, even for students who were not completely engaged in the design 
challenges. 

Principle 3: A robotics program should provide opportunities for students 
to explore the learning environment but at the same time encourage them to 
follow the process of scientific inquiry to complete design challenges. In this 
study, we found that some students needed unstructured time to explore LEGO 
robotics pieces and to play with various robot designs. Future research may be 
needed to examine how to maintain the balance between free play and problem 
solving structured by the scientific inquiry process. 

Principle 4: Students who have different levels of expertise with content 
knowledge and problem solving skills may need different activities in robotics 
programs. In our study, we found that structured robot building tasks worked 
well for a low performing student who had limited content knowledge and 
problem solving skills. Future studies may be needed to identify a variety of ac- 
tivities that may benefit students with different levels of ability. 

Principle 5: Long-term programs are needed to help students develop scien- 
tific inquiry skills over time. Students did not make significant progress in their 
scientific inquiry skills in this two-week program. We believe that part of the 
reasons might be that the intervention was not long enough. It might take years 
for students to acquire the skills. Longitudinal studies may be needed to inves- 
tigate how learners develop scientific inquiry skills over time and what support 
they need to develop the skills. 

Principle 6: A robotics program should prepare facilitators with knowledge 
and skills to facilitate scientific inquiry. In this study, we found that facilitators 
seldom mentioned scientific inquiry in their daily journal; we believe that their 
lack of scientific inquiry skills might have affected students’ learning. Future 
studies may be needed to research and develop effective strategies to train facili- 
tators on scientific inquiry skills. 
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